The three specular highlight models seen here are obviously more computationally expensive than diffuse lighting. But which is ultimately more expensive than the others?

The difference between Phong and Blinn is that Phong must compute the reflection vector, while Blinn computes the half-angle vector. The equation for computing the reflection vector is:

This involves a vector dot product, a scalar multiply, a vector-scalar multiply, and a vector addition (subtraction). Computing the half-angle vector requires doing a vector addition and performing a normalize operation. Normalizing a vector requires a vector dot product (dotting the vector with itself), taking the square-root of that value, and then a vector-scalar divide by that value.

Time once was that it was easy to know what was faster. The presence of a square-root operation alone would have condemned Blinn as the slower method. On modern 3D graphics hardware however, taking the reciprocal square-root (1 / √X) is generally about as fast as doing a vector multiply. This puts Blinn as approximately equal in performance to Phong; on some hardware, it may even be faster. In general, the performance difference between the two will be negligible.

Gaussian is a different story. It would be reasonable to expect the pow function, taking x^y for arbitrary values, to be slower than executing exp, e^x. They might have the same performance, but if one is going to be faster, it is more likely to be exp than pow. However, Gaussian also uses the inverse cosine to compute the angle between the normal and half-angle vector; that pretty much negates any possibility of performance parity. The inverse cosine computation is certainly not built into the hardware, and thus must be computed using the shader logic. And while this is likely true of exponentiation and power functions, Gaussian has to do two of these operations, compared to just one for Phong or Blinn.

One might consider using Gouraud shading with specular reflections as a method of optimization. That is, doing per-vertex specular reflections. Since there are fewer vertices than fragments, this might sound like a good idea. However, this is not for the best. Specular highlights do not interpolate linearly at all, so unless the mesh is finely divided, it will generally look awful.